SUMMARY

print(summarytools::dfSummary(df), method = 'render')

Data Frame Summary

df

Dimensions: 5000 x 18
Duplicates: 0

No	Variable	Stats / Values	Freqs (% of Valid)	Graph	Valid	Missing
1	account_length [numeric]	Mean (sd) : 100.3 (39.7) min ≤ med ≤ max: 1 ≤ 100 ≤ 243 IQR (CV) : 54 (0.4)	218 distinct values		5000 (100.0%)	0 (0.0%)
2	international_plan [factor]	1. 0 2. 1	4527 ( 90.5% ) 473 ( 9.5% )		5000 (100.0%)	0 (0.0%)
3	voice_mail_plan [factor]	1. 0 2. 1	3677 ( 73.5% ) 1323 ( 26.5% )		5000 (100.0%)	0 (0.0%)
4	number_vmail_messages [numeric]	Mean (sd) : 7.8 (13.5) min ≤ med ≤ max: 0 ≤ 0 ≤ 52 IQR (CV) : 17 (1.7)	48 distinct values		5000 (100.0%)	0 (0.0%)
5	total_day_minutes [numeric]	Mean (sd) : 180.3 (53.9) min ≤ med ≤ max: 0 ≤ 180.1 ≤ 351.5 IQR (CV) : 72.5 (0.3)	1961 distinct values		5000 (100.0%)	0 (0.0%)
6	total_day_calls [numeric]	Mean (sd) : 100 (19.8) min ≤ med ≤ max: 0 ≤ 100 ≤ 165 IQR (CV) : 26 (0.2)	123 distinct values		5000 (100.0%)	0 (0.0%)
7	total_day_charge [numeric]	Mean (sd) : 30.6 (9.2) min ≤ med ≤ max: 0 ≤ 30.6 ≤ 59.8 IQR (CV) : 12.3 (0.3)	1961 distinct values		5000 (100.0%)	0 (0.0%)
8	total_eve_minutes [numeric]	Mean (sd) : 200.6 (50.6) min ≤ med ≤ max: 0 ≤ 201 ≤ 363.7 IQR (CV) : 67.7 (0.3)	1879 distinct values		5000 (100.0%)	0 (0.0%)
9	total_eve_calls [numeric]	Mean (sd) : 100.2 (19.8) min ≤ med ≤ max: 0 ≤ 100 ≤ 170 IQR (CV) : 27 (0.2)	126 distinct values		5000 (100.0%)	0 (0.0%)
10	total_eve_charge [numeric]	Mean (sd) : 17.1 (4.3) min ≤ med ≤ max: 0 ≤ 17.1 ≤ 30.9 IQR (CV) : 5.8 (0.3)	1659 distinct values		5000 (100.0%)	0 (0.0%)
11	total_night_minutes [numeric]	Mean (sd) : 200.4 (50.5) min ≤ med ≤ max: 0 ≤ 200.4 ≤ 395 IQR (CV) : 67.8 (0.3)	1853 distinct values		5000 (100.0%)	0 (0.0%)
12	total_night_calls [numeric]	Mean (sd) : 99.9 (20) min ≤ med ≤ max: 0 ≤ 100 ≤ 175 IQR (CV) : 26 (0.2)	131 distinct values		5000 (100.0%)	0 (0.0%)
13	total_night_charge [numeric]	Mean (sd) : 9 (2.3) min ≤ med ≤ max: 0 ≤ 9 ≤ 17.8 IQR (CV) : 3.1 (0.3)	1028 distinct values		5000 (100.0%)	0 (0.0%)
14	total_intl_minutes [numeric]	Mean (sd) : 10.3 (2.8) min ≤ med ≤ max: 0 ≤ 10.3 ≤ 20 IQR (CV) : 3.5 (0.3)	170 distinct values		5000 (100.0%)	0 (0.0%)
15	total_intl_calls [numeric]	Mean (sd) : 4.4 (2.5) min ≤ med ≤ max: 0 ≤ 4 ≤ 20 IQR (CV) : 3 (0.6)	21 distinct values		5000 (100.0%)	0 (0.0%)
16	total_intl_charge [numeric]	Mean (sd) : 2.8 (0.7) min ≤ med ≤ max: 0 ≤ 2.8 ≤ 5.4 IQR (CV) : 0.9 (0.3)	170 distinct values		5000 (100.0%)	0 (0.0%)
17	number_customer_service_calls [numeric]	Mean (sd) : 1.6 (1.3) min ≤ med ≤ max: 0 ≤ 1 ≤ 9 IQR (CV) : 1 (0.8)	0 : 1023 ( 20.5% ) 1 : 1786 ( 35.7% ) 2 : 1127 ( 22.5% ) 3 : 665 ( 13.3% ) 4 : 252 ( 5.0% ) 5 : 96 ( 1.9% ) 6 : 34 ( 0.7% ) 7 : 13 ( 0.3% ) 8 : 2 ( 0.0% ) 9 : 2 ( 0.0% )		5000 (100.0%)	0 (0.0%)
18	churn [factor]	1. 0 2. 1	4293 ( 85.9% ) 707 ( 14.1% )		5000 (100.0%)	0 (0.0%)

Generated by summarytools 1.0.1 (R version 4.2.3)
2023-05-04

• We have no missing values -> perfect
• Heavily uneven counts of dependent variable (86 % no / 14 % yes) -> maybe sample for equality / maybe not because we loose information of other data
• Independent variables are on different scales -> standardize
• two (maybe three) categorical predictors: International plan / voice_mail_plan (/ maybe number_customer_service_calls) -> dummy encode -> not necessary as already 0 and 1
• Rest of data is numeric and most of the variables looks normally distributed with exception of number_vmail_messages and totat_intl_calls
  • transform these value to make them normal?
  • maybe make parts of them categorical? (recieving voice mail or not, calling internationally or not)
  • or maybe the categorical values that we have already give an indication for this
  • Test normality of variables
• Can variables be combined? We have day / eve / night / intl calls and for each of them minutes / calls / charge. Maybe we can combine this into one metric. Maybe average cost per minute or average cost per call?

CORRELATION PLOT BEFORE DATA ENGINEERING

df_numeric <- select_if(df, is.numeric)  # Subset numeric columns with dplyr

M <- cor(df_numeric)

p.mat <- cor_pmat(df_numeric)

ggcorrplot(M, hc.order = TRUE, type = "lower", lab = TRUE, p.mat = p.mat, sig.level=0.05, lab_size = 2, tl.cex = 10,outline.col = "white", ggtheme = ggplot2::theme_minimal(), colors = c("#2F4F4E", "white", "#0D8387")) 

Proves theory from before -> we can make one metric out of charge and minutes –> charge / minutes

DATA ENGINEERING

df$total_day_charge_per_minute <- ifelse(df$total_day_minutes == 0, 0, df$total_day_charge / df$total_day_minutes)
df$total_eve_charge_per_minute <- ifelse(df$total_eve_minutes == 0, 0, df$total_eve_charge / df$total_eve_minutes)
df$total_night_charge_per_minute <- ifelse(df$total_night_minutes == 0, 0, df$total_night_charge / df$total_night_minutes)
df$total_intl_charge_per_minute <- ifelse(df$total_intl_minutes == 0, 0, df$total_intl_charge / df$total_intl_minutes)
df <- subset(df, select = -c(total_day_charge, total_day_minutes, total_eve_charge, total_eve_minutes, total_night_charge, total_night_minutes, total_intl_charge, total_intl_minutes))

CORRELATION PLOT AFTER DATA ENGINEERING

df_numeric <- select_if(df, is.numeric)  # Subset numeric columns with dplyr

M <- cor(df_numeric)

p.mat <- cor_pmat(df_numeric)

ggcorrplot(M, hc.order = TRUE, type = "lower", lab = TRUE, p.mat = p.mat, sig.level=0.05, lab_size = 2, tl.cex = 10,outline.col = "white", ggtheme = ggplot2::theme_minimal(), colors = c("#2F4F4E", "white", "#0D8387")) 

Now we have non-correlated data

HIGHER ORDER FEATURES

Only squaring as we have no negative data. Cubing would be needed with negative data.

# squared
df2 <- df^2
df2 <- df2[,-c(2,3,10)]
colnames(df2) <- paste0(colnames(df2), '_sqd')

df <- cbind(df,df2)

Relationship between data in higher order

# theme_set(theme_minimal())
# 
# ggpairs(
#   data = df,
#   columns = c(1:9, 11:25),
#   mapping = aes(col = churn, alpha = .9)
# ) +
#   scale_fill_colorblind() +
#   scale_color_colorblind()

SAMPLING METHODS

As we have unbalanced data we need to use a sampling method to balance the classes. Hereby there are four different methods. OVER / UNDER / BOTH / ROSE.

library(ROSE)

FALSE Loaded ROSE 0.0-4

# OVER
df_OVER <- ovun.sample(churn~., data = df, method = "over")$data

table(df$churn)

FALSE 
FALSE    0    1 
FALSE 4293  707

table(df_OVER$churn)

FALSE 
FALSE    0    1 
FALSE 4293 4235

# UNDER
df_UNDER <- ovun.sample(churn~., data = df, method = "under")$data

table(df$churn)

FALSE 
FALSE    0    1 
FALSE 4293  707

table(df_UNDER$churn)

FALSE 
FALSE   0   1 
FALSE 672 707

# BOTH
df_BOTH <- ovun.sample(churn~., data = df, method = "both")$data

table(df$churn)

FALSE 
FALSE    0    1 
FALSE 4293  707

table(df_BOTH$churn)

FALSE 
FALSE    0    1 
FALSE 2524 2476

# ROSE
df_ROSE <- ROSE(churn ~ ., data = df, seed = 1, p = 0.5)$data

SAMPLING POST VISUALIZATION

# theme_set(theme_minimal())
# 
# ggpairs(
#   data = df_ROSE,
#   columns = c(1:9, 11:25),
#   mapping = aes(col = churn, alpha = .9)
# ) +
#   scale_fill_colorblind() +
#   scale_color_colorblind() +
#   labs(title = "Machine Learning Project")
# 
# ggpairs(
#   data = df_OVER,
#   columns = c(1:9, 11:25),
#   mapping = aes(col = churn, alpha = .9)
# ) +
#   scale_fill_colorblind() +
#   scale_color_colorblind() +
#   labs(title = "Machine Learning Project")

# ggpairs(
#   data = df_UNDER,
#   columns = c(1:9, 11:25),
#   mapping = aes(col = churn, alpha = .9)
# ) +
#   scale_fill_colorblind() +
#   scale_color_colorblind() +
#   labs(title = "Machine Learning Project")

# ggpairs(
#   data = df_BOTH,
#   columns = c(1:9, 11:25),
#   mapping = aes(col = churn, alpha = .9)
# ) +
#   scale_fill_colorblind() +
#   scale_color_colorblind() +
#   labs(title = "Machine Learning Project")

TRAIN AND TEST SPLIT

As we need to test the models we need to split the sampled data.

set.seed(1)
data <- df_OVER # choose which data to use df_ROSE / df_BOTH / df_UNDER / df_OVER / df
inds <- splitTools::partition(data$churn, p = c(train = 0.7, test = 0.3))
dftrain <- data[inds$train,]
dftest <- data[inds$test,]

SCALING

As some methods need scaled data we scale the data here to be centered.

norm.value <- preProcess(dftrain, method = c("center", "scale"))
dftrain <- predict(norm.value, dftrain)
dftest <- predict(norm.value, dftest)

NEURAL NET

# dftrain <- dftrain |> 
#   mutate_if(is.factor, as.character) |> 
#    mutate_if(is.character, as.numeric)
# 
# library(neuralnet)
# mod.neural <- neuralnet(churn ~ ., data = dftrain, hidden=c(15,15), linear.output = FALSE)
#  
# predicted.neural <- predict(mod.neural, dftest[,-c(10)])
# 
# confmat.neural <- confusionMatrix(data=predicted.neural, reference = dftest$churn, positive = '1')
#  
# CM_Function(confmat.neural)
#  
# roc_score.neural =roc(factor(dftest$churn, ordered=TRUE), factor(predicted.neural, ordered=TRUE))
# plot(roc_score.neural ,main ="ROC curve")

BOOSTING

set.seed(123)

# train bagged model
mod.boost <- boosting(churn ~., data=dftrain)

predicted.boost <- factor(predict(mod.boost, dftest, type="class")$class)

confmat.boost <- confusionMatrix(data=predicted.boost, reference = dftest$churn, positive = '1')

CM_Function(confmat.boost)

roc_score.boost =roc(factor(dftest$churn, ordered=TRUE), factor(predicted.boost, ordered=TRUE))

FALSE Setting levels: control = 0, case = 1

FALSE Setting direction: controls < cases

plot(roc_score.boost ,main ="ROC curve")

CTREE

tree_full <- rpart(churn ~ ., 
              data = dftrain, 
              method = "class",  # "class" because Y is a binary factor
              minbucket = 1,
              cp = 0.00001) 

# Plot tree
rpart.plot(tree_full, yesno = TRUE, digits =-6)

min_xerr<- which.min(tree_full$cptable[,"xerror"]) # select minimum cross-validation error
cp_bp <- tree_full$cptable[min_xerr,"CP"]  # find the corresponding CP value, to get the "best pruned " tree


mod.pruned_tree<- prune(tree_full, cp = cp_bp) # re-compute the tree with the selected Cp
rpart.plot(mod.pruned_tree, yesno = TRUE, digits =-3)

predicted.pruned_tree <- predict(mod.pruned_tree, dftest[,-c(10)], type = "class")

confmat.prunned_tree <- confusionMatrix(data=predicted.pruned_tree, reference = dftest$churn, positive = '1')

CM_Function(confmat.prunned_tree)

roc_score.prunned_tree =roc(factor(dftest$churn, ordered=TRUE), factor(predicted.pruned_tree, ordered=TRUE))

FALSE Setting levels: control = 0, case = 1

FALSE Setting direction: controls < cases

plot(roc_score.prunned_tree ,main ="ROC curve")

BAGGING

set.seed(123)
library(ipred)

FALSE 
FALSE Attaching package: 'ipred'

FALSE The following object is masked from 'package:adabag':
FALSE 
FALSE     bagging

library(pROC)

# train bagged model
ames_bag1 <- bagging(
  formula = churn ~ .,
  data = dftrain, 
  nbagg = 100,  
  coob = TRUE,
  control = rpart.control(minsplit = 2, cp = 0)
  )

ames_bag1

FALSE 
FALSE Bagging classification trees with 100 bootstrap replications 
FALSE 
FALSE Call: bagging.data.frame(formula = churn ~ ., data = dftrain, nbagg = 100, 
FALSE     coob = TRUE, control = rpart.control(minsplit = 2, cp = 0))
FALSE 
FALSE Out-of-bag estimate of misclassification error:  0.0439

predicted <- factor(ifelse(predict(ames_bag1, dftest[,-c(10)], type = 'prob')[,2] >= 0.5, 1, 0))
                    
CM_Function(confusionMatrix(data=predicted, reference = dftest$churn, positive = '1'))

roc_score=roc(factor(dftest$churn, ordered=TRUE), factor(predicted, ordered=TRUE)) #AUC score

FALSE Setting levels: control = 0, case = 1

FALSE Setting direction: controls < cases

auc <- round(auc(factor(dftest$churn, ordered=TRUE), factor(predicted, ordered=TRUE)),4)

FALSE Setting levels: control = 0, case = 1
FALSE Setting direction: controls < cases

ggroc(roc_score, colour = '#0D8387', size = 1) +
 ggtitle(paste0('ROC Curve ', '(AUC = ', auc, ')')) + theme_minimal() + theme(plot.title = element_text(face = "bold")) + labs(x="Specificity", y="Sensitivity")

KNN

set.seed(1)

df <- data.frame(k = seq(1, 30, 1), accuracy = rep(0, 30), sensitivity = rep(0, 30))

# iterating over different ks
for(i in 1:30){
  # nearest neighbor
  KNN1 <- knn3(y = dftrain$churn, x = dftrain[,-c(10)], k = i)

  # predictions response 
  KNN1.pred.valid.resp <- predict(KNN1, dftest[,-c(10)], type = "class")
  
  # predictions prob 
  KNN1.pred.valid.prob <- predict(KNN1, dftest[,-c(10)], type = "prob")[,2]
  
  # Confusionmatrix
  df$sensitivity[i] <- confusionMatrix(KNN1.pred.valid.resp, dftest$churn, positive = "1")$byClass[1]
  df$accuracy[i] <- confusionMatrix(KNN1.pred.valid.resp, dftest$churn, positive = "1")$overall[1]

}

# plot the k's
ggplot(df, aes(x=k)) + 
  geom_line(aes(y = sensitivity, colour = "Sensitivity")) + 
  geom_line(aes(y = accuracy, colour = "Accuracy")) + 
  labs(x = "Number of k nearest neighbours", 
       y = "Accuracy / Sensitivity", title = "Accuracy / Sensitivity regarding k") +
  theme_minimal() + 
  scale_y_continuous(name = "Sensitivity / Accuracy", limits = c(0.7, 1)) +
    scale_color_manual(name = "Values", values = c("Sensitivity" = "darkblue", "Accuracy" = "red")) + 
  xlim (1, 30)

mod.knn <- knn3(y = dftrain$churn, x = dftrain[,-c(10)], k = 2)

predicted.knn <- predict(mod.knn, dftest[,-c(10)], type = "class")

confmat.knn <- confusionMatrix(data=predicted.knn, reference = dftest$churn, positive = '1')

CM_Function(confmat.knn)

roc_score.qda =roc(factor(dftest$churn, ordered=TRUE), factor(predicted.knn, ordered=TRUE))

FALSE Setting levels: control = 0, case = 1

FALSE Setting direction: controls < cases

plot(roc_score.qda ,main ="ROC curve")

QDA

mod.qda <- qda(churn ~., data = dftrain)

predicted.qda <- predict(mod.qda, dftest[,-c(10)])$class

confmat.qda <- confusionMatrix(data=predicted.qda, reference = dftest$churn, positive = '1')

CM_Function(confmat.qda)

roc_score.qda =roc(factor(dftest$churn, ordered=TRUE), factor(predicted.qda, ordered=TRUE))

FALSE Setting levels: control = 0, case = 1

FALSE Setting direction: controls < cases

plot(roc_score.qda ,main ="ROC curve")

QLOG

mod.log <- glm(churn ~., data = dftrain, family = binomial(link = "probit"))

s <- step(mod.log)

FALSE Start:  AIC=6830.44
FALSE churn ~ account_length + international_plan + voice_mail_plan + 
FALSE     number_vmail_messages + total_day_calls + total_eve_calls + 
FALSE     total_night_calls + total_intl_calls + number_customer_service_calls + 
FALSE     total_day_charge_per_minute + total_eve_charge_per_minute + 
FALSE     total_night_charge_per_minute + total_intl_charge_per_minute + 
FALSE     account_length_sqd + number_vmail_messages_sqd + total_day_calls_sqd + 
FALSE     total_eve_calls_sqd + total_night_calls_sqd + total_intl_calls_sqd + 
FALSE     number_customer_service_calls_sqd + total_day_charge_per_minute_sqd + 
FALSE     total_eve_charge_per_minute_sqd + total_night_charge_per_minute_sqd + 
FALSE     total_intl_charge_per_minute_sqd
FALSE 
FALSE                                     Df Deviance    AIC
FALSE - account_length_sqd                 1   6780.4 6828.4
FALSE - total_day_calls                    1   6780.4 6828.4
FALSE - total_day_charge_per_minute        1   6780.5 6828.5
FALSE - total_day_charge_per_minute_sqd    1   6780.5 6828.5
FALSE - total_intl_charge_per_minute       1   6780.6 6828.6
FALSE - total_intl_charge_per_minute_sqd   1   6780.7 6828.7
FALSE - total_day_calls_sqd                1   6780.7 6828.7
FALSE - account_length                     1   6781.1 6829.1
FALSE - total_night_calls                  1   6781.1 6829.1
FALSE - total_night_calls_sqd              1   6781.3 6829.3
FALSE - total_eve_calls_sqd                1   6781.6 6829.6
FALSE - total_eve_calls                    1   6781.7 6829.7
FALSE <none>                                   6780.4 6830.4
FALSE - total_night_charge_per_minute_sqd  1   6782.7 6830.7
FALSE - total_night_charge_per_minute      1   6782.9 6830.9
FALSE - number_vmail_messages_sqd          1   6787.5 6835.5
FALSE - number_vmail_messages              1   6790.8 6838.8
FALSE - total_intl_calls_sqd               1   6792.4 6840.4
FALSE - total_intl_calls                   1   6798.9 6846.9
FALSE - number_customer_service_calls      1   6802.5 6850.5
FALSE - total_eve_charge_per_minute        1   6802.6 6850.6
FALSE - total_eve_charge_per_minute_sqd    1   6802.6 6850.6
FALSE - voice_mail_plan                    1   6807.8 6855.8
FALSE - number_customer_service_calls_sqd  1   6939.3 6987.3
FALSE - international_plan                 1   7403.1 7451.1
FALSE 
FALSE Step:  AIC=6828.44
FALSE churn ~ account_length + international_plan + voice_mail_plan + 
FALSE     number_vmail_messages + total_day_calls + total_eve_calls + 
FALSE     total_night_calls + total_intl_calls + number_customer_service_calls + 
FALSE     total_day_charge_per_minute + total_eve_charge_per_minute + 
FALSE     total_night_charge_per_minute + total_intl_charge_per_minute + 
FALSE     number_vmail_messages_sqd + total_day_calls_sqd + total_eve_calls_sqd + 
FALSE     total_night_calls_sqd + total_intl_calls_sqd + number_customer_service_calls_sqd + 
FALSE     total_day_charge_per_minute_sqd + total_eve_charge_per_minute_sqd + 
FALSE     total_night_charge_per_minute_sqd + total_intl_charge_per_minute_sqd
FALSE 
FALSE                                     Df Deviance    AIC
FALSE - total_day_calls                    1   6780.4 6826.4
FALSE - total_day_charge_per_minute        1   6780.5 6826.5
FALSE - total_day_charge_per_minute_sqd    1   6780.5 6826.5
FALSE - total_intl_charge_per_minute       1   6780.6 6826.6
FALSE - total_intl_charge_per_minute_sqd   1   6780.7 6826.7
FALSE - total_day_calls_sqd                1   6780.7 6826.7
FALSE - total_night_calls                  1   6781.1 6827.1
FALSE - total_night_calls_sqd              1   6781.3 6827.3
FALSE - total_eve_calls_sqd                1   6781.6 6827.6
FALSE - total_eve_calls                    1   6781.7 6827.7
FALSE <none>                                   6780.4 6828.4
FALSE - total_night_charge_per_minute_sqd  1   6782.7 6828.7
FALSE - total_night_charge_per_minute      1   6782.9 6828.9
FALSE - number_vmail_messages_sqd          1   6787.5 6833.5
FALSE - account_length                     1   6790.1 6836.1
FALSE - number_vmail_messages              1   6790.8 6836.8
FALSE - total_intl_calls_sqd               1   6792.4 6838.4
FALSE - total_intl_calls                   1   6798.9 6844.9
FALSE - number_customer_service_calls      1   6802.5 6848.5
FALSE - total_eve_charge_per_minute        1   6802.7 6848.7
FALSE - total_eve_charge_per_minute_sqd    1   6802.7 6848.7
FALSE - voice_mail_plan                    1   6807.8 6853.8
FALSE - number_customer_service_calls_sqd  1   6939.5 6985.5
FALSE - international_plan                 1   7403.4 7449.4
FALSE 
FALSE Step:  AIC=6826.44
FALSE churn ~ account_length + international_plan + voice_mail_plan + 
FALSE     number_vmail_messages + total_eve_calls + total_night_calls + 
FALSE     total_intl_calls + number_customer_service_calls + total_day_charge_per_minute + 
FALSE     total_eve_charge_per_minute + total_night_charge_per_minute + 
FALSE     total_intl_charge_per_minute + number_vmail_messages_sqd + 
FALSE     total_day_calls_sqd + total_eve_calls_sqd + total_night_calls_sqd + 
FALSE     total_intl_calls_sqd + number_customer_service_calls_sqd + 
FALSE     total_day_charge_per_minute_sqd + total_eve_charge_per_minute_sqd + 
FALSE     total_night_charge_per_minute_sqd + total_intl_charge_per_minute_sqd
FALSE 
FALSE                                     Df Deviance    AIC
FALSE - total_day_charge_per_minute        1   6780.5 6824.5
FALSE - total_day_charge_per_minute_sqd    1   6780.5 6824.5
FALSE - total_intl_charge_per_minute       1   6780.6 6824.6
FALSE - total_intl_charge_per_minute_sqd   1   6780.7 6824.7
FALSE - total_night_calls                  1   6781.1 6825.1
FALSE - total_night_calls_sqd              1   6781.3 6825.3
FALSE - total_eve_calls_sqd                1   6781.6 6825.6
FALSE - total_eve_calls                    1   6781.7 6825.7
FALSE <none>                                   6780.4 6826.4
FALSE - total_night_charge_per_minute_sqd  1   6782.7 6826.7
FALSE - total_night_charge_per_minute      1   6782.9 6826.9
FALSE - number_vmail_messages_sqd          1   6787.5 6831.5
FALSE - account_length                     1   6790.1 6834.1
FALSE - number_vmail_messages              1   6790.8 6834.8
FALSE - total_intl_calls_sqd               1   6792.4 6836.4
FALSE - total_day_calls_sqd                1   6794.1 6838.1
FALSE - total_intl_calls                   1   6798.9 6842.9
FALSE - number_customer_service_calls      1   6802.6 6846.6
FALSE - total_eve_charge_per_minute        1   6802.7 6846.7
FALSE - total_eve_charge_per_minute_sqd    1   6802.7 6846.7
FALSE - voice_mail_plan                    1   6807.9 6851.9
FALSE - number_customer_service_calls_sqd  1   6939.5 6983.5
FALSE - international_plan                 1   7404.3 7448.3
FALSE 
FALSE Step:  AIC=6824.47
FALSE churn ~ account_length + international_plan + voice_mail_plan + 
FALSE     number_vmail_messages + total_eve_calls + total_night_calls + 
FALSE     total_intl_calls + number_customer_service_calls + total_eve_charge_per_minute + 
FALSE     total_night_charge_per_minute + total_intl_charge_per_minute + 
FALSE     number_vmail_messages_sqd + total_day_calls_sqd + total_eve_calls_sqd + 
FALSE     total_night_calls_sqd + total_intl_calls_sqd + number_customer_service_calls_sqd + 
FALSE     total_day_charge_per_minute_sqd + total_eve_charge_per_minute_sqd + 
FALSE     total_night_charge_per_minute_sqd + total_intl_charge_per_minute_sqd
FALSE 
FALSE                                     Df Deviance    AIC
FALSE - total_intl_charge_per_minute       1   6780.7 6822.7
FALSE - total_intl_charge_per_minute_sqd   1   6780.8 6822.8
FALSE - total_night_calls                  1   6781.1 6823.1
FALSE - total_night_calls_sqd              1   6781.3 6823.3
FALSE - total_eve_calls_sqd                1   6781.6 6823.6
FALSE - total_eve_calls                    1   6781.7 6823.7
FALSE <none>                                   6780.5 6824.5
FALSE - total_night_charge_per_minute_sqd  1   6782.8 6824.8
FALSE - total_night_charge_per_minute      1   6782.9 6824.9
FALSE - number_vmail_messages_sqd          1   6787.6 6829.6
FALSE - account_length                     1   6790.2 6832.2
FALSE - number_vmail_messages              1   6790.9 6832.9
FALSE - total_intl_calls_sqd               1   6792.4 6834.4
FALSE - total_day_calls_sqd                1   6794.1 6836.1
FALSE - total_day_charge_per_minute_sqd    1   6798.1 6840.1
FALSE - total_intl_calls                   1   6799.0 6841.0
FALSE - number_customer_service_calls      1   6802.6 6844.6
FALSE - total_eve_charge_per_minute        1   6802.7 6844.7
FALSE - total_eve_charge_per_minute_sqd    1   6802.7 6844.7
FALSE - voice_mail_plan                    1   6807.9 6849.9
FALSE - number_customer_service_calls_sqd  1   6939.6 6981.6
FALSE - international_plan                 1   7404.3 7446.3
FALSE 
FALSE Step:  AIC=6822.67
FALSE churn ~ account_length + international_plan + voice_mail_plan + 
FALSE     number_vmail_messages + total_eve_calls + total_night_calls + 
FALSE     total_intl_calls + number_customer_service_calls + total_eve_charge_per_minute + 
FALSE     total_night_charge_per_minute + number_vmail_messages_sqd + 
FALSE     total_day_calls_sqd + total_eve_calls_sqd + total_night_calls_sqd + 
FALSE     total_intl_calls_sqd + number_customer_service_calls_sqd + 
FALSE     total_day_charge_per_minute_sqd + total_eve_charge_per_minute_sqd + 
FALSE     total_night_charge_per_minute_sqd + total_intl_charge_per_minute_sqd
FALSE 
FALSE                                     Df Deviance    AIC
FALSE - total_night_calls                  1   6781.3 6821.3
FALSE - total_night_calls_sqd              1   6781.5 6821.5
FALSE - total_eve_calls_sqd                1   6781.8 6821.8
FALSE - total_eve_calls                    1   6781.9 6821.9
FALSE <none>                                   6780.7 6822.7
FALSE - total_night_charge_per_minute_sqd  1   6782.9 6822.9
FALSE - total_night_charge_per_minute      1   6783.0 6823.0
FALSE - number_vmail_messages_sqd          1   6787.7 6827.7
FALSE - account_length                     1   6790.3 6830.3
FALSE - number_vmail_messages              1   6791.0 6831.0
FALSE - total_intl_calls_sqd               1   6792.6 6832.6
FALSE - total_intl_charge_per_minute_sqd   1   6794.1 6834.1
FALSE - total_day_calls_sqd                1   6794.3 6834.3
FALSE - total_day_charge_per_minute_sqd    1   6798.3 6838.3
FALSE - total_intl_calls                   1   6799.1 6839.1
FALSE - number_customer_service_calls      1   6802.8 6842.8
FALSE - total_eve_charge_per_minute        1   6802.8 6842.8
FALSE - total_eve_charge_per_minute_sqd    1   6802.8 6842.8
FALSE - voice_mail_plan                    1   6808.0 6848.0
FALSE - number_customer_service_calls_sqd  1   6939.6 6979.6
FALSE - international_plan                 1   7404.3 7444.3
FALSE 
FALSE Step:  AIC=6821.29
FALSE churn ~ account_length + international_plan + voice_mail_plan + 
FALSE     number_vmail_messages + total_eve_calls + total_intl_calls + 
FALSE     number_customer_service_calls + total_eve_charge_per_minute + 
FALSE     total_night_charge_per_minute + number_vmail_messages_sqd + 
FALSE     total_day_calls_sqd + total_eve_calls_sqd + total_night_calls_sqd + 
FALSE     total_intl_calls_sqd + number_customer_service_calls_sqd + 
FALSE     total_day_charge_per_minute_sqd + total_eve_charge_per_minute_sqd + 
FALSE     total_night_charge_per_minute_sqd + total_intl_charge_per_minute_sqd
FALSE 
FALSE                                     Df Deviance    AIC
FALSE - total_night_calls_sqd              1   6782.4 6820.4
FALSE - total_eve_calls_sqd                1   6782.4 6820.4
FALSE - total_eve_calls                    1   6782.6 6820.6
FALSE <none>                                   6781.3 6821.3
FALSE - total_night_charge_per_minute_sqd  1   6783.6 6821.6
FALSE - total_night_charge_per_minute      1   6783.7 6821.7
FALSE - number_vmail_messages_sqd          1   6788.3 6826.3
FALSE - account_length                     1   6790.8 6828.8
FALSE - number_vmail_messages              1   6791.6 6829.6
FALSE - total_intl_calls_sqd               1   6793.4 6831.4
FALSE - total_intl_charge_per_minute_sqd   1   6794.8 6832.8
FALSE - total_day_calls_sqd                1   6795.0 6833.0
FALSE - total_day_charge_per_minute_sqd    1   6799.3 6837.3
FALSE - total_intl_calls                   1   6799.9 6837.9
FALSE - total_eve_charge_per_minute        1   6803.2 6841.2
FALSE - total_eve_charge_per_minute_sqd    1   6803.2 6841.2
FALSE - number_customer_service_calls      1   6803.4 6841.4
FALSE - voice_mail_plan                    1   6808.6 6846.6
FALSE - number_customer_service_calls_sqd  1   6940.3 6978.3
FALSE - international_plan                 1   7404.5 7442.5
FALSE 
FALSE Step:  AIC=6820.37
FALSE churn ~ account_length + international_plan + voice_mail_plan + 
FALSE     number_vmail_messages + total_eve_calls + total_intl_calls + 
FALSE     number_customer_service_calls + total_eve_charge_per_minute + 
FALSE     total_night_charge_per_minute + number_vmail_messages_sqd + 
FALSE     total_day_calls_sqd + total_eve_calls_sqd + total_intl_calls_sqd + 
FALSE     number_customer_service_calls_sqd + total_day_charge_per_minute_sqd + 
FALSE     total_eve_charge_per_minute_sqd + total_night_charge_per_minute_sqd + 
FALSE     total_intl_charge_per_minute_sqd
FALSE 
FALSE                                     Df Deviance    AIC
FALSE - total_eve_calls_sqd                1   6783.5 6819.5
FALSE - total_eve_calls                    1   6783.7 6819.7
FALSE <none>                                   6782.4 6820.4
FALSE - total_night_charge_per_minute_sqd  1   6784.6 6820.6
FALSE - total_night_charge_per_minute      1   6784.8 6820.8
FALSE - number_vmail_messages_sqd          1   6789.5 6825.5
FALSE - account_length                     1   6791.9 6827.9
FALSE - number_vmail_messages              1   6792.7 6828.7
FALSE - total_intl_calls_sqd               1   6794.3 6830.3
FALSE - total_intl_charge_per_minute_sqd   1   6795.9 6831.9
FALSE - total_day_calls_sqd                1   6796.1 6832.1
FALSE - total_day_charge_per_minute_sqd    1   6800.6 6836.6
FALSE - total_intl_calls                   1   6800.7 6836.7
FALSE - number_customer_service_calls      1   6804.1 6840.1
FALSE - total_eve_charge_per_minute        1   6804.1 6840.1
FALSE - total_eve_charge_per_minute_sqd    1   6804.1 6840.1
FALSE - voice_mail_plan                    1   6809.8 6845.8
FALSE - number_customer_service_calls_sqd  1   6940.6 6976.6
FALSE - international_plan                 1   7406.6 7442.6
FALSE 
FALSE Step:  AIC=6819.52
FALSE churn ~ account_length + international_plan + voice_mail_plan + 
FALSE     number_vmail_messages + total_eve_calls + total_intl_calls + 
FALSE     number_customer_service_calls + total_eve_charge_per_minute + 
FALSE     total_night_charge_per_minute + number_vmail_messages_sqd + 
FALSE     total_day_calls_sqd + total_intl_calls_sqd + number_customer_service_calls_sqd + 
FALSE     total_day_charge_per_minute_sqd + total_eve_charge_per_minute_sqd + 
FALSE     total_night_charge_per_minute_sqd + total_intl_charge_per_minute_sqd
FALSE 
FALSE                                     Df Deviance    AIC
FALSE - total_eve_calls                    1   6783.8 6817.8
FALSE <none>                                   6783.5 6819.5
FALSE - total_night_charge_per_minute_sqd  1   6785.7 6819.7
FALSE - total_night_charge_per_minute      1   6785.9 6819.9
FALSE - number_vmail_messages_sqd          1   6790.6 6824.6
FALSE - account_length                     1   6793.1 6827.1
FALSE - number_vmail_messages              1   6793.9 6827.9
FALSE - total_intl_calls_sqd               1   6795.2 6829.2
FALSE - total_intl_charge_per_minute_sqd   1   6797.0 6831.0
FALSE - total_day_calls_sqd                1   6797.4 6831.4
FALSE - total_intl_calls                   1   6801.5 6835.5
FALSE - total_day_charge_per_minute_sqd    1   6801.8 6835.8
FALSE - number_customer_service_calls      1   6805.1 6839.1
FALSE - total_eve_charge_per_minute        1   6805.2 6839.2
FALSE - total_eve_charge_per_minute_sqd    1   6805.2 6839.2
FALSE - voice_mail_plan                    1   6810.9 6844.9
FALSE - number_customer_service_calls_sqd  1   6941.4 6975.4
FALSE - international_plan                 1   7408.7 7442.7
FALSE 
FALSE Step:  AIC=6817.84
FALSE churn ~ account_length + international_plan + voice_mail_plan + 
FALSE     number_vmail_messages + total_intl_calls + number_customer_service_calls + 
FALSE     total_eve_charge_per_minute + total_night_charge_per_minute + 
FALSE     number_vmail_messages_sqd + total_day_calls_sqd + total_intl_calls_sqd + 
FALSE     number_customer_service_calls_sqd + total_day_charge_per_minute_sqd + 
FALSE     total_eve_charge_per_minute_sqd + total_night_charge_per_minute_sqd + 
FALSE     total_intl_charge_per_minute_sqd
FALSE 
FALSE                                     Df Deviance    AIC
FALSE <none>                                   6783.8 6817.8
FALSE - total_night_charge_per_minute_sqd  1   6786.0 6818.0
FALSE - total_night_charge_per_minute      1   6786.1 6818.1
FALSE - number_vmail_messages_sqd          1   6790.9 6822.9
FALSE - account_length                     1   6793.5 6825.5
FALSE - number_vmail_messages              1   6794.2 6826.2
FALSE - total_intl_calls_sqd               1   6795.4 6827.4
FALSE - total_intl_charge_per_minute_sqd   1   6797.3 6829.3
FALSE - total_day_calls_sqd                1   6797.7 6829.7
FALSE - total_intl_calls                   1   6801.7 6833.7
FALSE - total_day_charge_per_minute_sqd    1   6802.1 6834.1
FALSE - number_customer_service_calls      1   6805.5 6837.5
FALSE - total_eve_charge_per_minute        1   6805.6 6837.6
FALSE - total_eve_charge_per_minute_sqd    1   6805.6 6837.6
FALSE - voice_mail_plan                    1   6811.3 6843.3
FALSE - number_customer_service_calls_sqd  1   6941.7 6973.7
FALSE - international_plan                 1   7408.7 7440.7

mod.log <- glm(s$formula, data = dftrain, family = binomial(link = "probit"))

predicted.log <- factor(ifelse(predict(mod.log, dftest[,-c(10)], type='response')>0.5,1,0))

confmat.log <- confusionMatrix(data=predicted.log, reference = dftest$churn, positive = '1')

CM_Function(confmat.log)

roc_score.log =roc(factor(dftest$churn, ordered=TRUE), factor(predicted.log, ordered=TRUE))

FALSE Setting levels: control = 0, case = 1

FALSE Setting direction: controls < cases

plot(roc_score.log ,main ="ROC curve")

GAUSSIAN SVM

library(e1071)
mod.svm  = svm(formula = churn ~ .,
               data = dftrain,
                 type = 'C-classification', # this is because we want to make a regression classification
                 kernel = 'radial')

predicted.svm <- predict(mod.svm, dftest[,-c(10)])

confmat.svm <- confusionMatrix(data=predicted.svm, reference = dftest$churn, positive = '1')

CM_Function(confmat.svm)

roc_score.svm =roc(factor(dftest$churn, ordered=TRUE), factor(predicted.svm, ordered=TRUE))

FALSE Setting levels: control = 0, case = 1

FALSE Setting direction: controls < cases

plot(roc_score.svm ,main ="ROC curve")

PAIRS PLOT

# Load the necessary libraries
library(ggplot2)
library(GGally)
library(ggthemes)

# Set the main color palette
colors <- c("#0D8387", "#870D27")

# Create the ggpair plot 
PAIRS1 <- ggpairs(df,columns = c(1:4), mapping = aes(col = `Customer Churn`, alpha = 0.9)) + scale_color_manual(values = colors) + scale_fill_manual(values = colors) + labs(title = "Customer Telecommunication Data", subtitle = "Customer Churn = Yes  is red", caption="From Variable 1 to 4") + theme(plot.title = element_text(face = "bold")) 

PAIRS1

FALSE `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
FALSE `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
FALSE `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
FALSE `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

PAIRS2 <- ggpairs(df,columns = c(5:9), mapping = aes(col = `Customer Churn`, alpha = 0.9)) + scale_color_manual(values = colors) + scale_fill_manual(values = colors) + labs(title = "Customer Telecommunication Data", subtitle = "Customer Churn = Yes  is red", caption="From Variable 5 to 9") + theme(plot.title = element_text(face = "bold"))

PAIRS2

PAIRS3 <- ggpairs(df,columns = c(10:10), mapping = aes(col = `Customer Churn`, alpha = 0.9)) + scale_color_manual(values = colors) + scale_fill_manual(values = colors) + labs(title = "Customer Telecommunication Data", subtitle = "", y="Count") + theme(plot.title = element_text(face = "bold")) + theme_minimal() + theme(plot.title = element_text(face = "bold")) + annotate("text", x = 2, y = 900, label = "14.14%", colour = "#870D27", size=8) + annotate("text", x = 1, y = 4500, label = "85.86%", colour = "#0D8387", size=8) + theme(axis.text.x=element_text(size=16))

PAIRS3

PAIRS4 <- ggpairs(df,columns = c(11:14), mapping = aes(col = `Customer Churn`, alpha = 0.9)) + scale_color_manual(values = colors) + scale_fill_manual(values = colors) + labs(title = "Customer Telecommunication Data", subtitle = "Customer Churn = Yes  is red",caption="From Variable 11 to 14") + theme(plot.title = element_text(face = "bold"))

PAIRS4

df$`Customer Churn` <- factor(df$`Customer Churn`, levels = c('No', 'Yes'), labels = c(0,1))

df$`Customer Churn` <- as.integer(df$`Customer Churn`)

df$`Customer Churn` <- df$`Customer Churn` -1 

Proportions_Churn <- sum(df$`Customer Churn`[df$`Customer Churn` == 1])/nrow(df)
Proportions_No_Churn <- 1-Proportions_Churn

Proportions of Customer who churned => 14.14% Versus 85.86% who didn’t churn.